Multi-modal search engines have experienced significant growth and widespread use in recent years, making them the second most common internet use. While search engine systems offer a range of services, the image search field has recently become a focal point in the information retrieval community, as the adage goes, "a picture is worth a thousand words". Although popular search engines like Google excel at image search accuracy and agility, there is an ongoing debate over whether their search results can be biased in terms of gender, language, demographics, socio-cultural aspects, and stereotypes. This potential for bias can have a significant impact on individuals' perceptions and influence their perspectives. In this paper, we present our study on bias and fairness in web search, with a focus on keyword-based image search. We first discuss several kinds of biases that exist in search systems and why it is important to mitigate them. We narrow down our study to assessing and mitigating occupational stereotypes in image search, which is a prevalent fairness issue in image retrieval. For the assessment of stereotypes, we take gender as an indicator. We explore various open-source and proprietary APIs for gender identification from images. With these, we examine the extent of gender bias in top-tanked image search results obtained for several occupational keywords. To mitigate the bias, we then propose a fairness-aware re-ranking algorithm that optimizes (a) relevance of the search result with the keyword and (b) fairness w.r.t genders identified. We experiment on 100 top-ranked images obtained for 10 occupational keywords and consider random re-ranking and re-ranking based on relevance as baselines. Our experimental results show that the fairness-aware re-ranking algorithm produces rankings with better fairness scores and competitive relevance scores than the baselines.

We propose an unsupervised method that, given a word, automatically selects non-abstract senses of that word from an online ontology and generates images depicting the corresponding entities. When faced with the task of learning a visual model based only on the name of an object, a common approach is to find images on the web that are associated with the object name, and then train a visual classifier from the search result. As words are generally polysemous, this approach can lead to relatively noisy models if many examples due to outlier senses are added to the model. We argue that images associated with an abstract word sense should be excluded when training a visual classifier to learn a model of a physical object. While image clustering can group together visually coherent sets of returned images, it can be difficult to distinguish whether an image cluster relates to a desired object or to an abstract sense of the word.

Gender-neutral internet searches yield results that nonetheless produce male-dominated output, finds a new study by a team of psychology researchers. Moreover, these search results have an effect on users by promoting gender bias and potentially influencing hiring decisions. The work, which appears in the journal Proceedings of the National Academy of Sciences (PNAS), is among the latest to uncover how artificial intelligence (AI) can alter our perceptions and actions. "There is increasing concern that algorithms used by modern AI systems produce discriminatory outputs, presumably because they are trained on data in which societal biases are embedded," says Madalina Vlasceanu, a postdoctoral fellow in New York University's Department of Psychology and the paper's lead author. "As a consequence, their use by humans may result in the propagation, rather than reduction, of existing disparities."

We are seeing more references to machine learning in how Google is ranking pages and other documents in search results. That seems to be a direction that will leave what we know as traditional, or old school signals that are referred to as ranking signals behind. It's still worth considering some of those older ranking signals because they may play a role in how things are ranked. As I was going through a new patent application from Google on ranking image search results, I decided that it was worth including what I used to look at when trying to rank images. Images can rank highly in image search, and they can also help pages that they appear upon rank higher in organic web results, because they can help make a page more relevant for the query terms that page may be optimized for.

We propose an unsupervised method that, given a word, automatically selects non-abstract senses of that word from an online ontology and generates images depicting the corresponding entities. When faced with the task of learning a visual model based only on the name of an object, a common approach is to find images on the web that are associated with the object name, and then train a visual classifier from the search result. As words are generally polysemous, this approach can lead to relatively noisy models if many examples due to outlier senses are added to the model. We argue that images associated with an abstract word sense should be excluded when training a visual classifier to learn a model of a physical object. While image clustering can group together visually coherent sets of returned images, it can be difficult to distinguish whether an image cluster relates to a desired object or to an abstract sense of the word.

Machine vision tools like facial recognition are increasingly being used for law enforcement, advertising, and other purposes. Pew Research Center itself recently used a machine vision system to measure the prevalence of men and women in online image search results. This kind of system develops its own rules for identifying men and women after seeing thousands of example images, but these rules can be hard for to humans to discern. To better understand how this works, we showed images of the Center's staff members to a trained machine vision system similar to the one we used to classify image searches. We then systematically obscured sections of each image to see which parts of the face caused the system to change its decision about the gender of the person pictured.

Melbourne man Milorad "Michael" Trkulja has won his high court battle to sue the search engine Google for defamation over images and search results that link him to the Melbourne criminal underworld. Trkulja said he would continue legal action against Google until it removed his name and photos from the internet. Trkulja, who was shot in the back in a Melbourne restaurant in 2004, successfully argued in the Victorian supreme court in 2012 that Google defamed him by publishing photos of him linked to hardened criminals of Melbourne's underworld. Four years later the Victorian court of appeal overturned the decision, finding the case had no prospect of successfully proving defamation. The high court disputed that ruling in a judgment on Wednesday and ordered Google to pay Trkulja's legal costs.

Microsoft wants to make finding just the right image online less of a crapshoot by using deep learning to help its Bing Image Search engine return more relevant results to users. Deep learning is a computationally intensive subset of machine learning that enables advanced analytical workloads. Using neural networks modeled after the human brain, deep learning systems can better mimic how people recognize patterns and process information. Peeling back the curtain a bit on the search engine's internal workings, the Bing Image Search Relevance Team at Microsoft revealed that they are using deep learning to unlock information held between the pixels of the billions of pictures found online. Microsoft's artificial intelligence (AI) searches for correlations between images and search queries mapped to semantic spaces used to derive meaning from data, even if the web pages containing those images lack any sort of text descriptors.

Google has introduced a change in how it presents image search results yesterday. Google has removed the convenient "view image" button from image search which allowed users to open the image alone instead of opening the website where the image was originally published. "Today we're launching some changes on Google Images to help connect users and useful websites. This will include removing the View Image button," Google said on its SearchLiaison Twitter page. "The Visit button remains, so users can see images in the context of the webpages they're on."

Google "construction worker" images and you'll see a lot of stock art, of men carrying lumber and standing in front of excavators with their arms crossed--even the Village People, if you scroll down far enough. But you'll have to search for something like "construction worker woman" to find more females in hard hats, including photos that look more like inspiration for a sexy Halloween costume. See the woman pulling her pigtail, two traffic cones on her chest a la Madonna? Or the one wearing a tight, low-cut top, hammer raised, hand on her hip? A few years ago, a team of researchers at the University of Washington wondered how image search results for occupations like construction worker and receptionist represented gender.